Autonomous Vehicle and Cloud Control System

ABSTRACT

The invention provides systems and methods for an Intelligent Road Infrastructure System (IRIS), which facilitates vehicle operations and control for connected automated vehicle highway (CAVH) systems. IRIS systems and methods provide vehicles with individually customized information and real-time control instructions for vehicle to fulfill the driving tasks such as car following, lane changing, and route guidance. IRIS systems and methods also manage transportation operations and management services for both freeways and urban arterials. In some embodiments, the IRIS comprises or consists of one of more of the following physical subsystems: (1) Roadside unit (RSU) network, (2) Traffic Control Unit (TCU) and Traffic Control Center (TCC) network, (3) vehicle onboard unit (OBU), (4) traffic operations centers (TOCs), and (5) cloud information and computing services. The IRIS manages one or more of the following function categories: sensing, transportation behavior prediction and management, planning and decision making, and vehicle control. IRIS is supported by real-time wired and/or wireless communication, power supply networks, and cyber safety and security services.

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 17/741,903, filed May 11, 2022, which is acontinuation of U.S. patent application Ser. No. 16/776,846, filed Jan.30, 2020, which is a continuation of U.S. patent application Ser. No.16/135,916, filed Sep. 19, 2018, now U.S. Pat. No. 10,692,365, issuedJun. 23, 2020, which claims priority to U.S. Provisional Pat. App. Ser.No. 62/627,005, filed Feb. 6, 2018 and is a continuation-in-part of andclaims priority to U.S. patent application Ser. No. 15/628,331, filedJun. 20, 2017, now U.S. Pat. No. 10,380,886, issued Aug. 13, 2019, whichclaims priority to U.S. Provisional Pat. App. Ser. No. 62/507,453, filedMay 17, 2017, each of which of the foregoing is incorporated herein byreference in its entirety.

FIELD

The present invention relates to an intelligent road infrastructuresystem providing transportation management and operations and individualvehicle control for connected and automated vehicles (CAV), and, moreparticularly, to a system controlling CAVs by sending individualvehicles with customized, detailed, and time-sensitive controlinstructions and traffic information for automated vehicle driving, suchas vehicle following, lane changing, route guidance, and other relatedinformation.

BACKGROUND

Autonomous vehicles, vehicles that are capable of sensing theirenvironment and navigating without or with reduced human input, are indevelopment. At present, they are in experimental testing and not inwidespread commercial use. Existing approaches require expensive andcomplicated on-board systems, making widespread implementation asubstantial challenge.

Alternative systems and methods that address these problems aredescribed in U.S. patent application Ser. No. 15/628,331, filed Jun. 20,2017, and U.S. Provisional Patent Application Ser. No. 62/626,862, filedFeb. 6, 2018, the disclosures which is herein incorporated by referencein its entirety (referred to herein as a CAVH system).

The invention provides systems and methods for an Intelligent RoadInfrastructure System (IRIS), which facilitates vehicle operations andcontrol for connected automated vehicle highway (CAVH) systems. IRISsystems and methods provide vehicles with individually customizedinformation and real-time control instructions for vehicle to fulfillthe driving tasks such as car following, lane changing, and routeguidance. IRIS systems and methods also manage transportation operationsand management services for both freeways and urban arterials.

SUMMARY

The invention provides systems and methods for an Intelligent RoadInfrastructure System (IRIS), which facilitates vehicle operations andcontrol for connected automated vehicle highway (CAVH) systems. IRISsystems and methods provide vehicles with individually customizedinformation and real-time control instructions for vehicle to fulfillthe driving tasks such as car following, lane changing, and routeguidance. IRIS systems and methods also manage transportation operationsand management services for both freeways and urban arterials.

In some embodiments, the IRIS comprises or consists of one of more ofthe following physical subsystems: (1) Roadside unit (RSU) network, (2)Traffic Control Unit (TCU) and Traffic Control Center (TCC) network, (3)vehicle onboard unit (OBU), (4) traffic operations centers (TOCs), and(5) cloud information and computing services. The IRIS manages one ormore of the following function categories: sensing, transportationbehavior prediction and management, planning and decision making, andvehicle control. IRIS is supported by real-time wired and/or wirelesscommunication, power supply networks, and cyber safety and securityservices.

The present technology provides a comprehensive system providing fullvehicle operations and control for connected and automated vehicle andhighway systems by sending individual vehicles with detailed andtime-sensitive control instructions. It is suitable for a portion oflanes, or all lanes of the highway. In some embodiments, thoseinstructions are vehicle-specific and they are sent by a lowest levelTCU, which are optimized and passed from a top level TCC. These TCC/TCUsare in a hierarchical structure and cover different levels of areas.

In some embodiments, provided herein are systems and methods comprising:an Intelligent Road Infrastructure System (IRIS) that facilitatesvehicle operations and control for a connected automated vehicle highway(CAVH). In some embodiments, the systems and methods provide individualvehicles with detailed customized information and time-sensitive controlinstructions for vehicle to fulfill the driving tasks such as carfollowing, lane changing, route guidance, and provide operations andmaintenance services for vehicles on both freeways and urban arterials.In some embodiments, the systems and methods are built and managed as anopen platform; subsystems, as listed below, in some embodiments, areowned and/or operated by different entities, and are shared amongdifferent CAVH systems physically and/or logically, including one ormore of the following physical subsystems:

-   -   a. Roadside unit (RSU) network, whose functions include sensing,        communication, control (fast/simple), and drivable ranges        computation;    -   b. Traffic Control Unit (TCU) and Traffic Control Center (TCC)        network;    -   c. Vehicle onboard units (OBU) and related vehicle interfaces;    -   d. Traffic operations centers; and    -   e. Cloud based platform of information and computing services.

In some embodiments, the systems and methods manage one or more of thefollowing function categories:

-   -   a. Sensing;    -   b. Transportation behavior prediction and management;    -   c. Planning and decision making; and    -   d. Vehicle control.

In some embodiments, the systems and methods are supported by one ormore of the following:

-   -   a. Real-time Communication via wired and wireless media;    -   b. Power supply network; and    -   c. Cyber safety and security system.

In some embodiments, the function categories and physical subsystems ofIRIS have various configurations in terms of function and physic deviceallocation. For example, in some embodiments a configuration comprises:

-   -   a. RSUs provide real-time vehicle environment sensing and        traffic behavior prediction, and send instantaneous control        instructions for individual vehicles through OBUs;    -   b. TCU/TCC and traffic operation centers provides short-term and        long-term transportation behavior prediction and management,        planning and decision making, and collecting/processing        transportation information with or without cloud information and        computing services;    -   c. The vehicle OBUs, as above, collect vehicle generated data,        such as vehicle movement and condition and send to RSUs, and        receive inputs from the RSUs. Based on the inputs from RSU, OBU        facilitates vehicle control. When the vehicle control system        fails, the OBU may take over in a short time period to stop the        vehicle safely. In some embodiments, the vehicle OBU contains        one or more of the following modules: (1) a communication        module, (2) a data collection module and (3) a vehicle control        module. Other modules may also be included.

In some embodiments, a communication module is configured for dataexchange between RSUs and OBUs, and, as desired, between other vehicleOBUs. Vehicle sourced data may include, but is not limit to:

-   -   a. Human input data, such as: origin-destination of the trip,        expected travel time, expected start and arrival time, and        service requests;    -   b. Human condition data, such as human behaviors and human        status (e.g., fatigue level); and    -   c. Vehicle condition data, such as vehicle ID, type, and the        data collected by the data collection module.

Data from RSUs may include, but is not limit to:

-   -   a. Vehicle control instructions, such as: desired longitudinal        and lateral acceleration rate, desired vehicle orientation;    -   b. Travel route and traffic information, such as: traffic        conditions, incident, location of intersection, entrance and        exit; and    -   c. Services data, such as: fuel station, point of interest.

In some embodiments, a data collection module collects data from vehicleinstalled external and internal sensors and monitors vehicle and humanstatus, including but not limited to one or more of:

-   -   a. Vehicle engine status;    -   b. Vehicle speed;    -   c. Surrounding objects detected by vehicles; and    -   d. Human conditions.

In some embodiments, a vehicle control module is used to execute controlinstructions from an RSU for driving tasks such as, car following andlane changing.

In some embodiments, the sensing functions of an IRIS generate acomprehensive information at real-time, short-term, and long-term scalefor transportation behavior prediction and management, planning anddecision-making, vehicle control, and other functions. The informationincludes but is not limited to:

-   -   a. Vehicle surrounding, such as: spacing, speed difference,        obstacles, lane deviation;    -   b. Weather, such as: weather conditions and pavement conditions;    -   c. Vehicle attribute data, such as: speed, location, type,        automation level;    -   d. Traffic state, such as: traffic flow rate, occupancy, average        speed;    -   e. Road information, such as: signal, speed limit; and    -   f. Incidents collection, such as: occurred crash and congestion.

In some embodiments, the IRIS is supported by sensing functions thatpredict conditions of the entire transportation network at variousscales including but not limited to:

-   -   a. Microscopic level for individual vehicles, such as:        longitudinal movements (car following, acceleration and        deceleration, stopping and standing), lateral movements (lane        keeping, lane changing);    -   b. Mesoscopic level for road corridor and segments, such as:        special event early notification, incident prediction, weaving        section merging and diverging, platoon splitting and        integrating, variable speed limit prediction and reaction,        segment travel time prediction, segment traffic flow prediction;        and    -   c. Macroscopic level for the road network, such as: potential        congestions prediction, potential incidents prediction, network        traffic demand prediction, network status prediction, network        travel time prediction.

In some embodiments, the IRIS is supported by sensing and predictionfunctions, realizes planning and decision-making capabilities, andinforms target vehicles and entities at various spacious scalesincluding, but not limited to:

-   -   a. Microscopic level, such as longitudinal control (car        following, acceleration and deceleration) and lateral control        (lane keeping, lane changing);    -   b. Mesoscopic level, such as: special event notification, work        zone, reduced speed zone, incident detection, buffer space, and        weather forecast notification. Planning in this level ensures        the vehicle follows all stipulated rules (permanent or        temporary) to improve safety and efficiency; and    -   c. Macroscopic level, such as: route planning and guidance,        network demand management.

In some embodiments, the planning and decision-making functions of IRISenhance reactive measures of incident management and support proactivemeasures of incident prediction and prevention, including but notlimited to:

-   -   a. For reactive measures, IRIS detects occurred incidents        automatically and coordinate related agencies for further        actions. It will also provide incident warnings and rerouting        instructions for affected traffic; and    -   b. For proactive measures, IRIS predicts potential incidents and        sends control instructions to lead affected vehicles to safety,        and coordinate related agencies for further actions.

In some embodiments, the IRIS vehicle control functions are supported bysensing, transportation behavior prediction and management, planning anddecision making, and further include, but are not limit to thefollowing:

-   -   a. Speed and headway keeping: keep the minimal headway and        maximal speed on the lane to reach the max possible traffic        capacity;    -   b. Conflict avoidance: detects potential accident/conflicts on        the lane, and then sends a warning message and conflict avoid        instructions to vehicles. Under such situations, vehicles must        follow the instructions from the lane management system;    -   c. Lane keeping: keep vehicles driving on the designated lane;    -   d. Curvature/elevation control: make sure vehicles keep and        adjust to the proper speed and angle based on factors such as        road geometry, pavement condition;    -   e. Lane changing control: coordinate vehicles lane changing in        proper orders, with the minimum disturbance to the traffic flow;    -   f. System boundary control: vehicle permission verification        before entering, and system takeover and handoff mechanism for        vehicle entering and exiting, respectively;    -   g. Platoon control and fleet management;    -   h. System failure safety measures: (1) the system provides        enough response time for a driver or the vehicle to take over        the vehicle control during a system fail, or (2) other measures        to stop vehicles safely; and    -   i. Task priority management: providing a mechanism to prioritize        various control objectives.

In some embodiments, the RSU has one or more module configurationsincluding, but not limited to:

-   -   a. Sensing module for driving environment detection;    -   b. Communication module for communication with vehicles, TCUs        and cloud via wired or wireless media;    -   c. Data processing module that processes the data from the        sensing and communication module;    -   d. Interface module that communicates between the data        processing module and the communication module; and    -   e. Adaptive power supply module that adjusts power delivery        according to the conditions of the local power grid with backup        redundancy.

In some embodiments, a sensing module includes one or more of theflowing types of sensors:

-   -   a. Radar based sensors that work with vision sensor to sense        driving environment and vehicle attribute data, including but        not limited to:        -   i. LiDAR;        -   ii. Microwave radar;        -   iii. Ultrasonic radar; and        -   iv. Millimeter radar;    -   b. Vision based sensors that work with radar based sensors to        provide driving environment data, including but not limited to:        -   i. Color camera;        -   ii. Infrared camera for night time; and        -   iii. Thermal camera for night time;    -   c. Satellite based navigation system that work with inertial        navigation system to support vehicle locating, including but not        limited to:        -   i. DGPS; and        -   ii. BeiDou System;    -   d. inertial navigation system that work with the satellite based        navigation system to support vehicle locating, including but not        limited to an inertial reference unit; and    -   e. Vehicle identification devices, including but not limited to        RFID.

In some embodiments, the RSUs are installed and deployed based onfunction requirements and environment factors, such as road types,geometry and safety considerations, including but not limited to:

-   -   a. Some modules are not necessarily installed at the same        physical location as the core modules of RSUs;    -   b. RSU spacing, deployment and installation methods may vary        based on road geometry to archive maximal coverage and eliminate        detection blind spots. Possible installation locations include        but not limited to: freeway roadside, freeway on/off ramp,        intersection, roadside buildings, bridges, tunnels, roundabouts,        transit stations, parking lots, railroad crossings, school        zones; and    -   c. RSU are installed on:        -   i. Fixed locations for long-term deployment; and        -   ii. Mobile platforms, including but not limited to: cars and            trucks, unmanned aerial vehicles (UAVs), for short-term or            flexible deployment.

In some embodiments, RSUs are deployed on special locations and timeperiods that require additional system coverage, and RSU configurationsmay vary. The special locations include, but are not limited to:

-   -   a. Construction zones;    -   b. Special events, such as sports games, street fairs, block        parties, concerts; and    -   c. Special weather conditions such as storms, heavy snow.

In some embodiments, the TCCs and TCUs, along with the RSUs, may have ahierarchical structure including, but not limited to:

-   -   a. Traffic Control Center (TCC) realizes comprehensive traffic        operations optimization, data processing and archiving        functionality, and provides human operations interfaces. A TCC,        based on the coverage area, may be further classified as        macroscopic TCC, regional TCC, and corridor TCC;    -   b. Traffic Control Unit (TCU), realizes real-time vehicle        control and data processing functionality, that are highly        automated based on preinstalled algorithms. A TCU may be further        classified as Segment TCU and point TCUs based on coverage        areas; and    -   c. A network of Road Side Units (RSUs), that receive data flow        from connected vehicles, detect traffic conditions, and send        targeted instructions to vehicles, wherein the point or segment        TCU can be physically combined or integrated with an RSU.

In some embodiments, the cloud based platform provides the networks ofRSUs and TCC/TCUs with information and computing services, including butnot limited to:

-   -   a. Storage as a service (STaaS), meeting additional storage        needs of IRIS;    -   b. Control as a service (CCaaS), providing additional control        capability as a service for IRIS;    -   c. Computing as a service (CaaS), providing entities or groups        of entities of IRIS that requires additional computing        resources; and    -   d. Sensing as a service (SEaaS), providing additional sensing        capability as a service for IRIS.

The systems and methods may include and be integrated with functions andcomponents described in U.S. Provisional Patent Application Ser. No.62/626,862, filed Feb. 6, 2018, herein incorporated by reference in itsentirety.

In some embodiments, the systems and methods provide a virtual trafficlight control function. In some such embodiments, a cloud-based trafficlight control system, characterized by including sensors in road sidesuch as sensing devices, control devices and communication devices. Insome embodiments, the sensing components of RSUs are provided on theroads (e.g, intersections) for detecting road vehicle traffic, forsensing devices associated with the cloud system over a networkconnection, and for uploading information to the cloud system. The cloudsystem analyzes the sensed information and sends information to vehiclesthrough communication devices.

In some embodiments, the systems and methods provide a traffic stateestimation function. In some such embodiments, the cloud system containsa traffic state estimation and prediction algorithm. A weighted datafusion approach is applied to estimate the traffic states, the weightsof the data fusion method are determined by the quality of informationprovided by sensors of RSU, TCC/TCU and TOC. When the sensor isunavailable, the method estimates traffic states on predictive andestimated information, guaranteeing that the system provides a reliabletraffic state under transmission and/or vehicle scarcity challenges.

In some embodiments, the systems and methods provide a fleet maintenancefunction. In some such embodiments, the cloud system utilizes itstraffic state estimation and data fusion methods to support applicationsof fleet maintenance such as Remote Vehicle Diagnostics, Intelligentfuel-saving driving and Intelligent charge/refuel.

In some embodiments, the IRIS contains high performance computationcapability to allocate computation power to realize sensing, prediction,planning and decision making, and control, specifically, at threelevels:

-   -   a. A microscopic level, typically from 1 to 10 milliseconds,        such as vehicle control instruction computation;    -   b. A mesoscopic level, typically from 10 to 1000 milliseconds,        such as incident detection and pavement condition notification;        and    -   c. macroscopic level, typically longer than 1 second, such as        route computing.

In some embodiments, the IRIS manages traffic and lane management tofacilitate traffic operations and control on various road facilitytypes, including but not limited to:

-   -   a. Freeway, with methods including but not limited to:        -   i. Mainline lane changing management;        -   ii. Traffic merging/diverging management, such as on-ramps            and off-ramps;        -   iii. High-occupancy/Toll (HOT) lanes;        -   iv. Dynamic shoulder lanes;        -   v. Express lanes;        -   vi. Automated vehicle penetration rate management for            vehicles at various automation levels; and        -   vii. Lane closure management, such as work zones, and            incidents; and    -   b. Urban arterials, with methods including but not limited to:        -   i. Basic lane changing management;        -   ii. Intersection management;        -   iii. Urban street lane closure management; and        -   iv. Mixed traffic management to accommodate various modes            such as bikes, pedestrians, and buses.

In some embodiments, the IRIS provides additional safety and efficiencymeasures for vehicle operations and control under adverse weatherconditions, including but not limited to:

-   -   a. High-definition map service, provided by local RSUs, not        requiring vehicle installed sensors, with the lane width, lane        approach(left/through/right), grade(degree of up/down), radian        and other geometry information;    -   b. Site-specific road weather information, provided by RSUs        supported the TCC/TCU network and the cloud services; and    -   c. Vehicle control algorithms designed for adverse weather        conditions, supported by site-specific road weather information.

In some embodiments, the IRIS includes security, redundancy, andresiliency measures to improve system reliability, including but notlimited to:

-   -   a. Security measures, including network security and physical        equipment security:        -   i. Network security measures, such as firewalls and            periodical system scan at various levels; and        -   ii. Physical equipment security, such as secured hardware            installation, access control, and identification tracker;    -   b. System redundancy. Additional hardware and software resources        standing-by to fill the failed counterparts;    -   c. System backup and restore, the IRIS system is backed up at        various intervals from the whole system level to individual        device level. If a failure is detected, recovery at the        corresponding scale is performed to restore to the closest        backup; and    -   d. System fail handover mechanism activated when a failure is        detected. A higher-level system unit identifies the failure and        performance corresponding procedure, to replace and/or restore        the failed unit.

Also provided herein are methods employing any of the systems describedherein for the management of one or more aspects of traffic control. Themethods include those processes undertaken by individual participants inthe system (e.g., drivers, public or private local, regional, ornational transportation facilitators, government agencies, etc.) as wellas collective activities of one or more participants working incoordination or independently from each other.

Some portions of this description describe the embodiments of theinvention in terms of algorithms and symbolic representations ofoperations on information. These algorithmic descriptions andrepresentations are commonly used by those skilled in the dataprocessing arts to convey the substance of their work effectively toothers skilled in the art. These operations, while describedfunctionally, computationally, or logically, are understood to beimplemented by computer programs or equivalent electrical circuits,microcode, or the like. Furthermore, it has also proven convenient attimes, to refer to these arrangements of operations as modules, withoutloss of generality. The described operations and their associatedmodules may be embodied in software, firmware, hardware, or anycombinations thereof.

Certain steps, operations, or processes described herein may beperformed or implemented with one or more hardware or software modules,alone or in combination with other devices. In one embodiment, asoftware module is implemented with a computer program productcomprising a computer-readable medium containing computer program code,which can be executed by a computer processor for performing any or allof the steps, operations, or processes described.

Embodiments of the invention may also relate to an apparatus forperforming the operations herein. This apparatus may be speciallyconstructed for the required purposes, and/or it may comprise ageneral-purpose computing device selectively activated or reconfiguredby a computer program stored in the computer. Such a computer programmay be stored in a non-transitory, tangible computer readable storagemedium, or any type of media suitable for storing electronicinstructions, which may be coupled to a computer system bus.Furthermore, any computing systems referred to in the specification mayinclude a single processor or may be architectures employing multipleprocessor designs for increased computing capability.

Embodiments of the invention may also relate to a product that isproduced by a computing process described herein. Such a product maycomprise information resulting from a computing process, where theinformation is stored on a non-transitory, tangible computer readablestorage medium and may include any embodiment of a computer programproduct or other data combination described herein.

DRAWINGS

FIG. 1 shows exemplary OBU Components. 101: Communication module: thatcan transfer data between RSU and OBU. 102: Data collection module: thatcan collect data of the vehicle dynamic and static state and generatedby human. 103: Vehicle control module: that can execute control commandfrom RSU. When the control system of the vehicle is damaged, it can takeover control and stop the vehicle safely. 104: Data of vehicle andhuman. 105: Data of RSU.

FIG. 2 shows an exemplary IRIS sensing framework. 201: Vehicles senddata collected within their sensing range to RSUs. 202: RSUs collectlane traffic information based on vehicle data on the lane; RSUsshare/broadcast their collected traffic information to the vehicleswithin their range. 203: RSU collects road incidents information fromreports of vehicles within its covering range. 204: RSU of the incidentsegment send incident information to the vehicle within its coveringrange. 205: RSUs share/broadcast their collected information of the lanewithin its range to the Segment TCUs. 206: RSUs collect weatherinformation, road information, incident information from the SegmentTCUs. 207/208: RSU in different segment share information with eachother. 209: RSUs send incident information to the Segment TCUs. 210/211:Different segment TCUs share information with each other. 212:Information sharing between RSUs and CAVH Cloud. 213: Informationsharing between Segment TCUs and CAVH Cloud.

FIG. 3 shows an exemplary IRIS prediction framework. 301: data sourcescomprising vehicle sensors, roadside sensors, and cloud. 302: datafusion module. 303: prediction module based on learning, statistical andempirical algorithms. 304: data output at microscopic, mesoscopic andmacroscopic levels.

FIG. 4 shows an exemplary Planning and Decision Making function. 401:Raw data and processed data for three level planning. 402: PlanningModule for macroscopic, mesoscopic, and microscopic level planning. 403:Decision Making Module for vehicle control instructions. 404 MacroscopicLevel Planning. 405 Mesoscopic Level Planning. 406 Microscopic LevelPlanning. 407 Data Input for Macroscopic Level Planning: raw data andprocessed data for macroscopic level planning. 408 Data Input forMesoscopic Level Planning: raw data and processed data for mesoscopiclevel planning. 409 Data Input for Microscopic Level Planning: raw dataand processed data for microscopic level planning.

FIG. 5 shows an exemplary vehicle control flow component. 501: Theplanning and prediction module send the information to control methodcomputation module. 502: Data fusion module receives the calculatedresults from different sensing devices. 503: Integrated data sent to thecommunication module of RSUs. 504: RSUs sends the control command to theOBUs.

FIG. 6 shows an exemplary flow chart of longitudinal control.

FIG. 7 shows an exemplary flow chart of latitudinal control.

FIG. 8 shows an exemplary flow chart of fail-safe control.

FIG. 9 shows exemplary RSU Physical Components. 901 CommunicationModule. 902 Sensing Module. 903 Power Supply Unit. 904 Interface Module:a module that communicates between the data processing module and thecommunication module. 905 Data Processing Module: a module thatprocesses the data. 909: Physical connection of Communication Module toData Processing Module. 910: Physical connection of Sensing Module toData Processing Module. 911: Physical connection of Data ProcessingModule to Interface Module. 912: Physical connection of Interface Moduleto Communication Module

FIG. 10 shows exemplary RSU internal data flows. 1001 CommunicationModule. 1002 Sensing Module. 1004 Interface Module: a module thatcommunicates between the data processing module and the communicationmodule. 1005 Data Processing Module. 1006 TCU. 1007 Cloud. 1008 OBU.1013: Data flow from Communication Module to Data Processing Module.1014: Data flow from Data Processing Module to Interface Module. 1015:Data flow from Interface Module to Communication Module. 1016: Data flowfrom Sensing Module to Data Processing Module.

FIG. 11 shows an exemplary TCC/TCU Network Structure. 1101: controltargets and overall system information provided by macroscopic TCC toregional TCC. 1102: regional system and traffic information provided byregional TCC to macroscopic TCC. 1103: control targets and regionalinformation provided by regional TCC to corridor TCC. 1104: corridorsystem and traffic information provided by corridor TCC to regional TCC.1105: control targets and corridor system information provided bycorridor TCC to segment TCU. 1106: segment system and trafficinformation provided by segment TCU to corridor TCC. 1107: controltargets and segment system information provided by segment TCU to pointTCU. 1108: point system and traffic information provided by point TCU tocorridor TCU. 1109: control targets and local traffic informationprovided by point TCU to RSU. 1110: RSU status and traffic informationprovided by RSU to point TCU. 1111: customized traffic information andcontrol instructions from RSU to vehicles. 1112: information provided byvehicles to RSU. 1113: the services provided by the cloud to RSU/TCC-TCUnetwork.

FIG. 12 shows an exemplary architecture of a cloud system.

FIG. 13 shows an exemplary IRIS Computation Flowchart. 1301: DataCollected From RSU, including but not limited to image data, video data,radar data, On-board unit data. 1302: Data Allocation Module, allocatingcomputation resources for various data processing. 1303 ComputationResources Module for actual data processing. 1304 GPU, graphicprocessing unit, mainly for large parallel data. 1305 CPU, centralprocessing unit, mainly for advanced control data. 1306 Predictionmodule for IRIS prediction functionality. 1307 Planning module for IRISplanning functionality. 1308 Decision Making for IRIS decision-makingfunctionality. 1309 data for processing with computation resourceassignment. 1310 processed data for prediction module, planning module,decision making module. 1311 results from prediction module to planningmodule. 1312 results from planning module to decision making module.

FIG. 14 shows an exemplary Traffic and Lane Management Flowchart. 1401Lane management related data collected by RSU and OBU. 1402 Controltarget and traffic information from upper level IRIS TCU/TCC network.1403 Lane management and control instructions.

FIG. 15 shows an exemplary Vehicle Control in Adverse Weather component.1501: vehicle status, location and sensor data. 1502: comprehensiveweather and pavement condition data and vehicle control instructions.1503: wide area weather and traffic information obtained by the TCU/TCCnetwork.

FIG. 16 shows an exemplary IRIS System Security Design. 1601: Networkfirewall. 1602: Internet and outside services. 1603: Data center fordata services, such as data storage and processing. 1604: Local server.1605: Data transmission flow.

FIG. 17 shows an exemplary IRIS System Backup and Recovery component.1701: Cloud for data services and other services. 1702: Intranet. 1703:Local Storage for backup. 1704: any IRIS devices, i.e. RSU, TCU, or TCC.

FIG. 18 shows an exemplary System Failure Management component.

FIG. 19 shows a sectional view of an exemplary RSU deployment.

FIG. 20 shows a top view of an exemplary RSU deployment.

FIG. 21 shows exemplary RSU lane management on a freeway segment.

FIG. 22 shows exemplary RSU lane management on a typical urbanintersection.

DETAILED DESCRIPTION

Exemplary embodiments of the technology are described below. It shouldbe understood that these are illustrative embodiments and that theinvention is not limited to these particular embodiments.

FIG. 1 shows an exemplary OBU containing a communication module 101, adata collection module 102, and a vehicle control module 103. The datacollection module 102 collects data related to a vehicle and a human 104and then sends it 104 to an RSU through communication module 101. Also,OBU can receive data of RSU 105 through communication module 101. Basedon the data of RSU 105, the vehicle control module 103 helps control thevehicle.

FIG. 2 illustrates an exemplary framework of a lane management sensingsystem and its data flow.

The RSU exchanges information between the vehicles and the road andcommunicates with TCUs, the information including weather information,road condition information, lane traffic information, vehicleinformation, and incident information.

FIG. 3 illustrates exemplary workflow of a basic prediction process of alane management sensing system and its data flow. In some embodiments,fused multi-source data collected from vehicle sensors, roadside sensorsand the cloud is processed through models including but not limited tolearning based models, statistical models, and empirical models. Thenpredictions are made at different levels including microscopic,mesoscopic, and macroscopic levels using emerging models includinglearning based, statistic based, and empirical models.

FIG. 4 shows exemplary planning and decision making processes in anIRIS. Data 401 is fed into planning module 402 according to threeplanning level respectively 407, 408, and 409. The three planningsubmodules retrieve corresponding data and process it for their ownplanning tasks. In a macroscopic level 404, route planning and guidanceoptimization are performed. In a mesoscopic level 405, special event,work zone, reduced speed zone, incident, buffer space, and extremeweather are handled. In a microscopic level 406, longitudinal controland lateral control are generated based on internal algorithm. Aftercomputing and optimization, all planning outputs from the three levelsare produced and transmitted to decision making module 403 for furtherprocessing, including steering, throttle control, and braking.

FIG. 5 shows exemplary data flow of an infrastructure automation basedcontrol system. The control system calculates the results from allsensing detectors, conducts data fusion, and exchanges informationbetween RSUs and Vehicles. The control system comprises: a) ControlMethod Computation Module 501; b) Data Fusion Module 502; c)Communication Module (RSU) 503; and d) Communication Module (OBU) 504.

FIG. 6 illustrates an exemplary process of vehicle longitudinal control.As shown in the figure, vehicles are monitored by the RSUs. If relatedcontrol thresholds (e.g., minimum headway, maximum speed, etc.) arereached, the necessary control algorithms is triggered. Then thevehicles follow the new control instructions to drive. If instructionsare not confirmed, new instructions are sent to the vehicles.

FIG. 7 illustrates an exemplary process of vehicle latitudinal control.As shown in the figure, vehicles are monitored by the RSUs. If relatedcontrol thresholds (e.g., lane keeping, lane changing, etc.) arereached, the necessary control algorithms are triggered. Then thevehicles follows the new control instructions to drive. If instructionsare not confirmed, new instructions are sent to the vehicles.

FIG. 8 illustrates an exemplary process of vehicle fail safe control. Asshown in the figure, vehicles are monitored by the RSUs. If an erroroccurs, the system sends the warning message to the driver to warn thedriver to control the vehicle. If the driver does not make any responseor the response time is not appropriate for driver to take the decision,the system sends the control thresholds to the vehicle. If relatedcontrol thresholds (e.g., stop, hit the safety equipment, etc.) arereached, the necessary control algorithms is triggered. Then thevehicles follows the new control instructions to drive. If instructionsare not confirmed, new instructions are sent to the vehicles.

FIG. 9 shows an exemplary physical component of a typical RSU,comprising a Communication Module, a Sensing Module, a Power SupplyUnit, an Interface Module, and a Data Processing Module. The RSU may anyof variety of module configurations. For example, for the sense module,a low cost RSU may only include a vehicle ID recognition unit forvehicle tracking, while a typical RSU includes various sensors such asLiDAR, cameras, and microwave radar.

FIG. 10 shows an exemplary internal data flow within a RSU. The RSUexchanges data with the vehicle OBUs, upper level TCU and the cloud. Thedata processing module includes two processors: external objectcalculating Module (EOCM) and AI processing unit. EOCM is for trafficobject detection based on inputs from the sensing module and the AIprocessing unit focuses more on decision-making processes.

FIG. 11 show an exemplary structure of a TCC/TCU network. A macroscopicTCC, which may or may not collaborate with an external TOC, manages acertain number of regional TCCs in its coverage area. Similar, aregional TCC manages a certain number of corridor TCCs, a corridor TCCmanages a certain number of segment TCUs, a segment TCU manages acertain number of point TCUs, and a point TCUs manages a certain numberof RSUs. An RSU sends customized traffic information and controlinstructions to vehicles and receives information provided by vehicles.The network is supported by the services provided by the cloud.

FIG. 12 shows how an exemplary cloud system communicates with sensors ofRSU, TCC/TCU (1201) and TOC through communication layers (1202). Thecloud system contains cloud infrastructure (1204), platform (1205), andapplication service (1206). The application services also support theapplications (1203).

FIG. 13 shows exemplary data collected from sensing module 1301 such asimage data, video data, and vehicle status data. The data is dividedinto two groups by the data allocation module1302: large parallel dataand advanced control data. The data allocation module 1302 decides howto assign the data 1309 with the computation resources 1303, which aregraphic processing units (GPUs) 1304 and central processing units (CPUs)1305. Processed data 1310 is sent to prediction 1306, planning 1307, anddecision making modules 1308. The prediction module provides results tothe planning module 1311, and the planning module provides results 1312to the decision making module.

FIG. 14 shows how exemplary data collected from OBUs and RSUs togetherwith control targets and traffic information from upper level IRISTCC/TCC network 1402 are provided to a TCU. The lane management moduleof a TCU produces lane management and vehicle control instructions 1403for a vehicle control module and lane control module.

FIG. 15 shows exemplary data flow for vehicle control in adverseweather. Table 1, below, shows approaches for measurement of adverseweather scenarios.

TABLE 1 IRIS Measures for Adverse Weather Scenarios IRIS HDMap + TOC +Normal autonomous vehicle(only RSU(Camera + Radar + Lidar)/OBU cansensors) greatly mitigate the impact of adverse Camera Radar Lidarweather. Impact in Visibility Detecting Detecting Solution SolutionEnhancement adverse of distance distance for degrade for degrade forweather lines/signs/ degraded. degraded. of of distance vehicle objectsvisibility. detection. control. degraded. Rain ** ** ** HDMap RSU has aRSU can Snow *** ** ** provides whole control Fog **** **** **** info ofvision of vehicle Sandstorm **** **** **** lane/line/sign/ all vehiclesaccording geometry, on the to weather which road, so the (e.g., lowerenhance chance of the speed RSU's crash with on icy vision. other road).vehicles are eliminated. Number of “*” means the degree of decrease.

FIG. 16 shows exemplary IRIS security measures, including networksecurity and physical equipment security. Network security is enforcedby firewalls 1601 and periodically complete system scans at variouslevels. These firewalls protect data transmission 1605 either betweenthe system and an Internet 1601 or between data centers 1603 and localservers 1604. For physical equipment security, the hardware is safelyinstalled and secured by an identification tracker and possiblyisolated.

In FIG. 17, periodically, IRIS system components 1704 back up the datato local storage 1703 in the same Intranet 1702 through firewall 1601.In some embodiments, it also uploads backup copy through firewall 1601to the Cloud 1701, logically locating in the Internet 1702.

FIG. 18 shows an exemplary periodic IRIS system check for systemfailure. When failure happens, the system fail handover mechanism isactivated. First, failure is detected and the failed node is recognized.The functions of failed node are handed over to shadow system andsuccess feedback is sent back to an upper level system if nothing goeswrong. Meanwhile, a failed system/subsystem is restarted and/orrecovered from a most recent backup. If successful, feedback is reportedto an upper level system. When the failure is addressed, the functionsare migrated back to the original system.

Exemplary hardware and parameters that find use in embodiments of thepresent technology include, but are not limited to the following:

-   OBU:    -   a) Communication module Technical Specifications        -   Standard Conformance: IEEE 802.11p-2010        -   Bandwidth: 10 MHz        -   Data Rates: 10 Mbps        -   Antenna Diversity CDD Transmit Diversity        -   Environmental Operating Ranges: −40° C. to +55° C.        -   Frequency Band: 5 GHz        -   Doppler Spread: 800 km/h        -   Delay Spread: 1500 ns        -   Power Supply: 12/24V    -   b) Data collection module Hardware technical Specifications        -   Intuitive PC User Interface for functions such as            configuration, trace, transmit, filter, log etc.        -   High data transfer rate    -   c) Software technical Specifications        -   Tachograph Driver alerts and remote analysis.        -   Real-Time CAN BUS statistics.        -   CO2 Emissions reporting.    -   d) Vehicle control module Technical Specifications        -   Low power consumption        -   Reliable longitudinal and lateral vehicle control-   RSU Design    -   a) communication module which include three communication        channels:        -   Communication with vehicles including DSRC/4G/5G (e.g., MK5            V2X from Cohda Wireless)        -   Communication with point TCUs including wired/wireless            communication (e.g., Optical Fiber from Cablesys)        -   Communication with cloud including wired/wireless            communication with at least 20M total bandwidth    -   b) data Processing Module which include two processors:        -   External Object Calculating Module (EOCM)            -   Process Object detection using Data from the sensing                module and other necessary regular calculation (e.g.,                Low power fully custom ARM/X86 based processor)        -   AI processing Unit            -   Machine learning            -   Decision making/planning and prediction processing    -   c) an interface Module:        -   FPGA based Interface unit        -   FPGA processor that acts like a bridge between the AI            processors and the External Object Calculating Module            processors and send instructions to the communication            modules-   The RSU deployment    -   a. Deployment location    -   The RSU deployment is based on function requirement and road        type. An RSU is used for sensing, communicating, and controlling        vehicles on the roadway to provide automation. Since the LIDAR        and other sensors (like loop detectors) need different special        location, some of them can be installed separately from the core        processor of RSU.    -   Two exemplary types of RSU location deployment type:        -   i. Fixed location deployment. The location of this type of            RSU are fixed, which is used for serving regular roadways            with fixed traffic demand on the daily basis.        -   ii. Mobile deployment. Mobile RSU can be moved and settled            in new place and situation swiftly, is used to serve            stochastic and unstable demand and special events, crashes,            and others. When an event happens, those mobile RSU can be            moved to the location and perform its functions.    -   b. Method for coverage    -   The RSUs may be connected (e.g., wired) underground. RSUs are        mounted on poles facing down so that they can work properly. The        wings of poles are T-shaped. The roadway lanes that need CAVH        functions are covered by sensing and communication devices of        RSU. There are overlaps between coverage of RSUs to ensure the        work and performance.    -   c. Deployment Density    -   The density of deployment depends on the RSU type and        requirement. Usually, the minimum distance of two RSU depends on        the RSU sensors with minimum covering range.    -   d. Blind spot handling    -   There may be blind sensing spots causing by vehicles blocking        each other. The issue is common and especially serious when        spacing between vehicles are close. A solution for this is to        use the collaboration of different sensing technologies from        both RSUs deployed on infrastructures and OBUs that are deployed        on vehicles.    -   This type of deployment is meant to improve traffic condition        and control performance, under certain special conditions.        Mobile RSU can be brought by agents to the deployment spot. In        most cases, due to the temporary use of special RSUs, the poles        for mounting are not always available. So, those RSU may be        installed on temporary frames, buildings along the roads, or        even overpasses that are location-appropriate.

Certain exemplary RSU configurations are shown in FIGS. 19-22. FIG. 19shows a sectional view of an exemplary RSU deployment. FIG. 20 shows anexemplary top view of an RSU deployment. In this road segment, sensingis covered by two types of RSU: 901 RSU A: camera groups, the mostcommonly used sensors for objects detection; and 902 RSU B: LIDARgroups, which makes 3D representation of targets, providing higheraccuracy. Cameras sensor group employ a range that is lower than LIDAR,e.g. in this particular case, below 150 m, so a spacing of 150 m alongthe roads for those camera groups. Other type of RSUs have lessrequirement on density (e.g., some of them like LIDAR or ultrasonicsensors involve distances that can be greater).

FIG. 21 shows an exemplary RSU lane management configuration for afreeway segment. The RSU sensing and communication covers each lane ofthe road segment to fulfill the lane management functions examples(showed in red arrows in figure) including, but not limited to: 1) Lanechanging from one lane to another; 2) Merging manipulations from anonramp; 3) Diverging manipulations from highway to offramp; 4) Weavingzone management to ensure safety; and 5) Revisable lane management.

FIG. 22 shows an exemplary lane management configuration for a typicalurban intersection. The RSU sensing and communication covers each cornerof the intersection to fulfill the lane management functions examples(showed in red in figure) including: 1) Lane changing from one lane toanother; 2) Movement management (exclusive left turns in at this lane);3) Lane closure management at this leg; and 4) Exclusive bicycle lanemanagement.

We claim:
 1. A system comprising a road side unit (RSU) network thatcomprises a plurality of networked communication devices spaced along aroadway, wherein the RSU network is configured to communicate with: a) atraffic control unit (TCU) that communicates with and managesinformation from a plurality of RSU networks and communicates with andis managed by a traffic control center (TCC); and b) on board units(OBUs) of a plurality of vehicles traveling on said roadway.
 2. Thesystem of claim 1 wherein said RSU network is configured to sendvehicle-specific control instructions to vehicle OBUs.
 3. The system ofclaim 2 wherein said control instructions comprise commands for carfollowing, lane changing, lane keeping, longitudinal speed, lateralspeed, vehicle orientation, acceleration, deceleration, and/or routeguidance.
 4. The system of claim 1 wherein said RSU network isconfigured to sense vehicles on a road.
 5. The system of claim 1 whereineach RSU of the RSU network comprises a sensing module, a communicationmodule, a data processing module, an interface module, and/or anadaptive power supply module.
 6. The system of claim 1 wherein each RSUof the RSU network comprises a radar based sensor, a vision basedsensor, a satellite based navigation system component, and/or a vehicleidentification component.
 7. The system of claim 1 wherein the RSUs ofthe RSU network are deployed at spacing intervals within the range of 50to 500 meters.
 8. The system of claim 1 wherein said RSU network isconfigured to provide high-resolution maps comprising lane width, laneapproach, grade, and road geometry information to vehicles.
 9. Thesystem of claim 1 wherein said RSU network is configured to collectinformation comprising weather information, road condition information,lane traffic information, vehicle information, and/or incidentinformation and broadcast said information to vehicles and/or to the TCUnetwork.
 10. The system of claim 1 wherein said RSU network isconfigured to communicate with a cloud database.
 11. The system of claim1 wherein said RSU network is configured to provide data to OBUs, saiddata comprising vehicle control instructions, travel route and trafficinformation, and/or services data.
 12. The system of claim 1 whereinsaid RSU network comprises RSUs installed at one or more fixed locationsselected from the group consisting of a freeway roadside, freeway on/offramp, intersection, roadside building, bridge, tunnel, roundabout,transit station, parking lot, railroad crossing, and/or school zone. 13.The system of claim 1 wherein said RSU network comprises RSUs installedat one or more mobile platforms selected from the group consisting ofvehicles and unmanned aerial drones.
 14. The system of claim 1 whereinsaid RSU network is configured to communicate with said TCU network inreal-time over wired and/or wireless channels.
 15. The system of claim 1wherein said RSU network is configured to communicate with said OBUs inreal-time over wireless channels.
 16. The system of claim 6 wherein saidsatellite based navigation system component is configured to communicatewith OBUs and locate vehicles.